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We study the dynamics of macroscopic observables such as the magnetization and the energy 
per degree of freedom in Ising spin models on random graphs of finite connectivity, with random 
bonds and/or heterogeneous degree distributions. To do so we generalize existing implementations 
of dynamical replica theory and cavity field techniques to systems with strongly disordered and 
locally tree-like interactions. We illustrate our results via application to the dynamics of e.g. ±J 
spin-glasses on random graphs and of the overlap in finite connectivity Sourlas codes. All results 
are tested against Monte Carlo simulations. 
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Recent years have witnessed a surge of interest in the 
study of finitely connected disordered spin systems. From 
a physical point of view, despite their lack of a realistic 
geometry and their mean-field nature, the finite number 
of neighbours per spin in such models does give rise to a 
non-trivial local geometry and ensuing artifacts. Here we 
simply regard the random bond finite connectivity Ising 
spin system as the archetypal interacting particle model 
on a disordered random graph. Such models are impor- 
tant in the understanding of algorithmic complexity in 
theoretical computer science 0,13, 01 , and also underly 
recent theoretical advances for an important class of er- 
ror correcting codes 0, 0, El ■ It has been shown that 
the tuning of the degree distribution and/or the connec- 
tivity strengths in complex networks can lead to atypi- 
cal mean field critical phenomena 0, IM > an d they are 
now increasingly and fruitfully used for modeling neu- 
ral, social, intern et, gen e regulatory and proteomic net- 
works [lCJ, |ll|, |l2|, |l3|, |l4j . While our quantitative under- 
standing of the equilibrium properties of these systems 
is quite advanced (see e.g. llfi|). the tools available 
for studying their non-equilibrium behaviour are com- 
paratively poor. There has been so me p rogress in apply- 
ing the path integral techniques of |lj| to spherical and 
related models 1 181. a nd to Ising models with parallel 
spin updating [lfj, |20j . Generalizing such approaches to 
Ising spin models with Glauber-type dynamics requires 
the treatment of non-trivial functional order parameters 
which have, as yet, not been adequately controlled. An 
alternative approach, which we follow here, is to gener- 
alize the techniques of dynamical replica theory (DRT) 
[2~l1 l22j | . together with the cavity field concept, to finitely 
connected disordered spin systems. This approach has 
already proven fruitful for weakly disordered dilute fcr- 
romagnets [2j| where each spin is effectively in an iden- 
tical environment. In this letter, in contrast, we study 
the dynamics of strongly disordered versions of finitely 



connected Ising systems, where each spin is in a highly 
heterogeneous environment, due to the presence of either 
random bonds or nodes with variable degrees. 

Our model consists of N Ising spins Sj G {— 1, 1}, i = 
1, . . . , N, whose mutual interactions are characterized by 
a range-free symmetric adjacency matrix with entries 
Cij G {0,1} and symmetric bonds Jy G H. We de- 
fine cu = 0, and draw the bond strengths Jy i.i.d. from 
some distribution Q{J). The probability of finding any 
state s = (si, . . . , sn) of the system at time t is given 
by the measure pt{s), which evolves as the spins align 
asynchronously and stochastically to their local fields, ac- 
cording to a Glauber dynamics in the form of the master 
equation 



A' 



dt 



Pt{s) = ^2[p t (F k s)w k {F k s) -p t (s)w k (s)] (1) 



fc=i 



where F k s = (si, . . . , —s k , ■ ■ ■ , sjv) is the fcth spin-flip 
operator and the transition rates w k (s) have the standard 
form 

u>fc(s) = \ {1 - s fe tanh[/3/i fc (s)]} (2) 



with the local fields hi(s) = J2j^i c ij^ij s j + This 
process evolves toward equilibrium, with the Boltzmann 
measure and with Hamiltonian 



ff(s) = -£\ 

i<j 



(3) 



Following the procedure outlined for fully connected 
systems |2lJ, |22( we consider the evolution of two macro- 
scopic observables, the magnetization m(s) = N^ 1 J^i s i 
and the internal energy e(s) = N z2i<j c ijJij s i s j- 
We will abbreviate i~2 = (m,e). One easily derives 
a Kramers-Moyal (KM) expansion for their probability 
density Pt(fi) = S s Pt(s)<$[fi — ^( s )]- O n finite times 



2 



one finds that only the Liouville term in the KM expan- 
sion survives the thermodynamic limit, so the observables 
(m, e) evolve deterministically, i.e. limjv— ►«> ft(O) — 
5[Sl — n t ] with 



d 

dt 



n t = iim V Pt (s|n t )yV(s)[n(Fis)-n(s)] (4) 

N— >oo * — 4 z — » 

s i 

Pt (s)5[n - n(s)] 



p t (s\fl) 



(5) 



Equation J2J still involves the conditional microscopic 
distribution p t (s\fl). To proceed we follow 0,113 : we 
(i) assume that the observables £1 are self-averaging at 
all times (which one expects to be true) , and (ii) approx- 
imate the microscopic measure p t (s\£l) by the maximum 
entropy distribution given the condition that the macro- 
scopic observables take the value ft. These assumptions 
imply that our observables evolve according to 



m t = —m t + I dh T>(h\m,t,et)ta.nh(/3h) (6) 



—e t = -2e t - j dh T>(h\m t , e t )hta,nh(/3h) (7) 

Here T>(h\m t , et) denotes the asymptotic distribution of 
local fields in a system with magnetization mt and energy 
et, 



V{h\e tl mt) = lim 

N— »cx 



1 N 



-Ms)]) r 



(8) 



where the average (. . .) m e is over the maximum entropy 
distribution given the values of the observables, viz. over 



p(s|m t ,e t ) 



5[m t - m(s)]S[e t - e(s)] 
J2 S > S[mt - m(s')]S[e t - e(s')] 



(9) 



and [. . .]dis is over the disorder (the realization of the 
random graph and bonds). 

The relatively simple solution given in |23( can be un- 
derstood within the current framework. Since all sites 
were identical, there was just a single cavity field and 
hence the distribution of local fields was uniform across 
sites and could be given explicitly in terms of the observ- 
ables. 

The field distribution T>(h\m t , et) is readily calculated, 
even in the presence of bond or degree disorder, either 
via the replica method or via the cavity approach for di- 
lute systems [l5| (in the microcanonical or the canonical 
framework, respectively) j33| . Here the resulting equa- 
tions from either approach are the same. They cor- 
respond to the maximum entropy distribution, given 
(mt,e t ), which equals the equilibrium distribution of a 
system with Hamiltonian (|3J| but with a pseudo inverse 
temperature (3 and pseudo external field 9/(3. The latter 
act as Lagrange parameters, enforcing the condition that 
the equilibrium distribution gives the required values of 
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FIG. 1: Evolution of the magnetization m (top figures) 
and the energy e (bottom figures), for Ising spins on a 3- 
regular random graph with random bonds, and with time 
measured in units of updates per spin. Bond distribution: 
Q(J) = 7?<5(J - 1) + (1 - rj)5{J + 1). Solid lines denote the 
theoretical predictions. Dotted lines represent the simulation 
data (system size N = 10,000 and averaged over 50 runs), 
with dot-dashed lines giving the averages ± 1 standard devi- 
ation. Left pictures: r\ = 0.95 and j3 = 0.65. Right pictures: 
77 = 0.97 and = 1.2. 

{mt, et). The prefix 'pseudo' indicates that these param- 
eters need not be physical: there could be states (mt, e*) 
for which (5 is negative. Within the cavity formalism we 
can work either with the ensemble or with a particular 
realization of the disorder. The latter tends to be nu- 
merically simpler, due to the inherent (finite size) noise 
in population dynamics in the ensemble, which limits the 
accuracy with which the Lagrange parameters can be cal- 
culated. Working in areas of phase space where replica 
symmetry is expected to be exact and where belief prop- 
agation converges on any given graph realization, it is 
possible to find the Lagrange parameters to high preci- 
sion. For large graph sizes the differences between results 
for graph realization and the ensemble average ought to 
vanish. 

The resulting numerical algorithm is as follows. At 
any given point in time we know the instantaneous val- 
ues (m t ,e t ) of our observables. We then run a belief 
propagation algorithm on our graph, for a given pair of 
Lagrange parameters 9) which act as inverse temper- 
ature and external field; once the belief propagation has 
converged we can measure (m t (/3 , 9) , e t (/3 , 9)) . We now 
vary the Lagrange parameters and repeat the above until 
we satisfy the condition 



mt = mt(j3,9), e t = e t 0,9) 



(10) 



Finally we use the cavity fields generated with the correct 
values of ($, 9) to give the local field distribution within 
our graph, with which we can evaluate the force terms in 
(IfTl . 
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FIG. 2: Evolution of the inverse pseudo-temperature La- 
grange parameter for the experiments shown in figure 
Clearly, the relaxation of f3 need not be monotonic. 

In figure ^ we compare the results of our analysis 
with Monte Carlo simulations for a ±J spin-glass on 
a 3-regular graph. We sampled our graph uniformly 
from all connected graphs where each site has exactly 
three neighbours and each bond is drawn i.i.d. from 
Q(J) = rjS(J - 1) + (1 - T])5(J + 1). All simulations 
were carried out with a system size of N = 10,000, and 
were run on the same realization of the graph as the cav- 
ity field calculations. We see an excellent correspondence 
between theory and simulations. We have taken r\ to be 
relatively large (predominance of ferromagnetic bonds), 
since we did not wish to move into a region where belief 
propagation would not converge, a condition one expects 
to be closely related to instability in the AT sense |2J,|25j. 
In such regions it is no longer possible to use belief propa- 
gation for accurately evaluating the Lagrange parameters 
/3 and 6. 

It is also of interest to note in figure that the evolu- 
tion of the pseudo-temperature need not be monotonic. 
Assuming that the location of the AT line [24| is similar 
to that of the fully connected case, i.e. that it goes contin- 
uously from the zero-temperature instability (T = 0, rj = 
Y5) [ll El to the triple point (T » 1.13, 77 « 0.85), as 
shown in |25j , one could envisage a situation where start- 
ing from a replica symmetric (RS) phase, the parameters 
(/3, rj) could be chosen such that the final equilibrium 
phase was also RS, but where the dynamics would take 
the system through a regime of phase space where in 
equilibrium one would find replica symmetry breaking. 
Since there, the belief propagation (or any other replica 
symmetric) algorithm would not converge in a time of 
O(N), the algorithm would become stuck en route to RS 
equilibrium. 

In figure |3] we examine the order parameter flow in 
ferromagnetic random graphs with average connectivity 
2 and 3, respectively. Here each bond is independently 
defined to be present (cjj = 1) or absent (cy = 0) with 
probability c/N, where c is the average connectivity, lead- 
ing to a graph with a Poisson degree distribution (or 
Erdos-Renyi graph) . In these systems the inhomogencity 
of the local environment of the spins is no longer caused 
by bond disorder, but by non-uniform connectivity. The 
agreement between theory and simulations in the case 
c = 2 is significantly worse than that in the other ex- 
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FIG. 3: Evolution of the magnetization m (top) and the 
energy e (bottom), for Ising spins on a Poisson random graph 
(of average connectivity c) with ferromagnetic bonds Jij — 1, 
and with time measured in units of updates per spin (top 4 
graphs) or flow inm-e place (bottom 2 graphs). Solid lines 
denote the theoretical predictions. Dotted lines represent the 
simulation data (system size N = 10,000 and averaged over 50 
runs), with dot-dashed lines giving the averages ± 1 standard 
deviation. Left pictures: c = 2 and T = 0.75. Right pictures: 
c — S and T — 2.8. 

amples presented. Here the maximum entropy measure 
appears to be a much less accurate approximation of the 
true microscopic distribution, which tells us that the sys- 
tem evolves through statistically a-typical microscopic 
states, and predicts a relaxation of the order parame- 
ters that is far too quick. This would appear to be re- 
lated to the increased heterogeneity associated with lower 
temperatures and lower average connectivity. However, 
plotting in the m — e plane we see that the predicted 
direction of the flow is still quite reasonable. 

As a final example we turn to the de-coding dynamics 
of finite connectivity Sourlas codes 0, 0, with 2-body 
interactions, which can easily be studied within the cur- 
rent framework. In particular, in figure 0] we consider 
the case of an unbiased source broadcasting through a 
binary symmetric channel with flip probability 0.04 and 
rate | (the channel capacity as given by Shannon's theo- 
rem is 0.76...). If a message (£1, . . . , £/v) is sent across this 
channel, and our estimator for this message (given the 
corrupt channel) is given by (£1, . . . ,£jv), then a natural 
performance measure is the overlap between the message 
sent and the decoded message, O = N^Y^i&ti- We 
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FIG. 4: Decoding dynamics of the overlap O (left) and the 
magnetization m (right), for a 2-body interaction and rate | 
Sourlas error correcting code. Solid lines denote the theoret- 
ical predictions. Dotted lines represent the simulation data 
(system size N = 10,000 and averaged over 50 runs), with 
dot-dashed lines giving the averages ± 1 standard deviation. 
The temperature is Nishimori's temperature for the flip prob- 
ability (error rate) 0.04. 

decode at Nishimori's temperature, which is the tem- 
perature maximizing this particular overlap observable 
|29j (the so-called maximizer of posterior marginals). Al- 
though qualitatively correct, the predicted relaxation of 
the order parameters is again too fast compared with the 
simulation data. 

In this letter we have presented a relatively simple 
dynamical formalism, combining dynamical replica the- 
ory with the cavity method, to be used as a system- 
atic approximation tool with which to understand the 
main features of the dynamics of dilute and disordered 
spin systems. We regard the wide applicability of the 
method as its strength. From the various applications 
presented here we see that our approach performs excel- 
lently in some cases, but relaxes too quickly in others, 
compared with numerical simulations. This is not un- 
expected |2lj |. However, as with the original dynamical 
replica theory, there is scope for increasing the order pa- 
rameter set jU H3 , which should improve its accuracy 
systematically, albeit at a numerical cost. At present our 
method requires the convergence of belief propagation. 
It would therefore seem that breaking replica symmetry 
within this formalism will be non-trivial to implement, 
even though theoretically it is a straightforward general- 
ization. For a single experiment we here run belief propa- 
gation O(10 5 ) times; running a finite temperature 1RSB 
scheme that many times would be computationally ex- 
tremely demanding without further approximations. 

We warmly thank M. Okada, B. Wemmenhove and 
T. Nikoletopoulos for helpful discussions and comments. 
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